The present invention relates to methods and apparatus for displaying images, and more particularly, to display methods and apparatus which utilize multiple displaced portions of an output device, e.g., liquid crystal display, to represent an image.
Color display devices have become the principal display devices of choice for most computer users. The display of color on a monitor is normally achieved by operating the display device to emit light, e.g., a combination of red, green, and blue light, which results in one or more colors being perceived by a human viewer.
In cathode ray tube (CRT) display devices, the different colors of light are generated via the use of phosphor coatings which may be applied as dots in a sequence on the screen of the CRT. A different phosphor coating is normally used to generate each of the three colors, red, green, and blue resulting in repeating sequences of phosphor dots which, when excited by a beam of electrons, will generate the colors red, green and blue.
The term pixel is commonly used to refer to one spot in, e.g., a rectangular grid of thousands of such spots. The spots are individually used by a computer to form an image on the display device. For a color CRT, where a single triad of red, green and blue phosphor dots cannot be addressed, the smallest possible pixel size will depend on the focus, alignment and bandwidth of the electron guns used to excite the phosphors. The light emitted from one or more triads of red, green and blue phosphor dots, in various arrangements known for CRT displays, tend to blend together giving, at a distance, the appearance of a single colored light source.
In color displays, the intensity of the light emitted corresponding to the additive primary colors, red, green and blue, can be varied to get the appearance of almost any desired color pixel. Adding no color, i.e., emitting no light, produces a black pixel. Adding 100 percent of all three colors results in white.
FIG. 1 illustrates a known portable computer 100, which comprises a housing 101, a disk drive 105, keyboard 104 and a flat panel display 102.
Portable personal computers 100 tend to use liquid crystal displays (LCD) or other flat panel display devices 102, as opposed to CRT displays. This is because flat panel displays tend to be small and lightweight as compared to CRT displays. In addition, flat panel displays tend to consume less power than comparably sized CRT displays making them better suited for battery powered applications than CRT displays.
As the quality of flat panel color displays continues to increase and their cost decreases, flat panel displays are beginning to replace CRT displays in desktop applications. Accordingly, flat panel displays, and LCDs in particular, are becoming ever more common.
Over the years, most image processing techniques, including the generation and display of fonts, e.g., sets of characters, on computer screens, have been developed and optimized for display on CRT display devices.
Unfortunately, existing text display routines fail to take into consideration the unique physical characteristics of flat panel display devices. These physical characteristics differ considerably from the characteristics of CRT devices particularly in regard to the physical characteristics of the RGB color light sources.
Color LCD displays are exemplary of display devices which utilize multiple distinctly addressable elements, referred to herein as pixel sub-elements or pixel sub-components, to represent each pixel of an image being displayed. Normally, each pixel on a color LCD display is represented by a single pixel element which usually comprises three non-square elements, i.e., red, green and blue (RGB) pixel sub-components. Thus, a set of RGB pixel sub-components together make up a single pixel element. LCD displays of the known type comprise a series of RGB pixel sub-components which are commonly arranged to form stripes along the display. The RGB stripes normally run the entire length of the display in one direction. The resulting RGB stripes are sometimes referred to as xe2x80x9cRGB stripingxe2x80x9d. Common LCD monitors used for computer applications, which are wider than they are tall, tend to have RGB stripes running in the vertical direction.
FIG. 2A illustrates a known LCD screen 200 comprising a plurality of rows (R1-R12) and columns (C1-C16) which may be used as the display 102. Each row/column intersection forms a square which represents one pixel element. FIG. 2B illustrates the upper left hand portion of the known display 200 in greater detail.
Note in FIG. 2B how each pixel element, e.g., the (R1, C4) pixel element, comprises three distinct sub-element or sub-components, a red sub-component 206, a green sub-component 207 and a blue sub-component 208. Each known pixel sub-component 206, 207, 208 is ⅓ or approximately ⅓ the width of a pixel while being equal, or approximately equal, in height to the height of a pixel. Thus, when combined, the three ⅓ width pixel sub-components 206, 207, 208 form a single pixel element.
As illustrated in FIG. 2A, one known arrangement of RGB pixel sub-components 206, 207, 208 form what appear to be vertical color stripes down the display 200. Accordingly, the arrangement of ⅓ width color sub-components 206, 207, 208, in the known manner illustrated in FIGS. 2A and 2B, is sometimes called xe2x80x9cvertical stripingxe2x80x9d.
While only 12 rows and 16 columns are shown in FIG. 2A for purposes of illustration, common columnxc3x97row ratios include, e.g., 640xc3x97480, 800xc3x97600, and 1024xc3x97768. Note that known display devices normally involve the display being arranged in landscape fashion, i.e., with the monitor being wider than it is high as illustrated in FIG. 2A, and with stripes running in the vertical direction.
LCDs are are manufactured with pixel sub-components arranged in several additional patterns including, e.g., zig-zags and a delta pattern common in camcorder view finders. While features of the present invention can be used with such pixel sub-component arrangements, since the RGB striping configuration is more common, the exemplary embodiments of the present invention will be explained in the context of using RGB striped displays.
Traditionally, each set of pixel sub-components for a pixel element is treated as a single pixel unit. Accordingly, in known systems luminous intensity values for all the pixel sub-components of a pixel element are generated from the same portion of an image. Consider for example, the image represented by the grid 220 illustrated in FIG. 2C. In FIG. 2C each square represents an area of an image which is to be represented by a single pixel element, e.g., a red, green and blue pixel sub-component of the corresponding square of the grid 230. In FIG. 2C a shaded circle is used to represent a single image sample from which luminous intensity values are generated. Note how a single sample 222 of the image 220 is used in known systems to generate the luminous intensity values for each of the red, green, and blue pixel sub-components 232, 233, 234. Thus, in known systems, the RGB pixel sub-components are generally used as a group to generate a single colored pixel corresponding to a single sample of the image to be represented.
The light from each pixel sub-component group effectively adds together to create the effect of a single color whose hue, saturation, and intensity depend on the value of each of the three pixel sub-components. Say, for example, each pixel sub-component has a potential intensity of between 0 and 255. If all three pixel sub-components are given 255 intensity, the pixel will be perceived as being white. However, if all three pixel sub-components are given a value turning off each of the three pixel sub-components, the pixel will be perceived as black. By varying the respective intensities of each pixel sub-component, it is possible to generate millions of colors in between these two extremes.
Since, in the known system a single sample is mapped to a triple of pixel sub-components which are each ⅓ of a pixel in width, spatial displacement of the left and right pixel sub-components occurs since the centers of these elements are ⅓ from the center of the sample.
Consider for example that an image to be represented was a red cube with green and blue components equal to zero. As a result of the displacement between the sample and green image sub-component, when displayed on an LCD display of the type illustrated in FIG. 2A, the apparent position of the cube on the display will be shifted ⅓ of a pixel to the left of its actual position. Similarly, a blue cube would appear to be displaced ⅓ of a pixel to the right. Thus, known imaging techniques used with LCD screens can result in undesirable image displacement errors.
Text characters represent one type of image which is particularly difficult to accurately display given typical flat panel display resolutions of 72 or 96 dots (pixels) per inch (dpi). Such display resolutions are far lower than the 600 dpi supported by most printers and the even higher resolutions found in most commercially printed text such as books and magazines.
Because of the relatively low display resolution of most video display devices, not enough pixels are available to draw smooth character shapes, especially at common text sizes of 10, 12, and 14 point type. At such common text rendering sizes, gradations between different sizes and weights, e.g., the thickness, of the same typeface, are far coarser than their print equivalent.
The relatively coarse size of standard pixels tends to create aliasing effects which give displayed type characters jagged edges. For example, the coarse size of pixels tends to result in the squaring off of serifs, the short lines or ornaments at the ends, e.g., bottom, of strokes which form a typeface character. This makes it difficult to accurately display many highly readable or ornamental typefaces which tend to use serifs extensively.
Such problems are particularly noticeable in the stems, e.g., thin vertical portions, of characters. Because pixels are the minimum display unit of conventional monitors, it is not possible to display stems of characters using conventional techniques with less than one pixel stem weight. Furthermore, stem weight can only be increased a pixel at a time. Thus, stem weights leap from one to two pixels wide. Often one pixel wide character stems are too light, while two pixel wide character stems are too bold. Since creating a boldface version of a typeface on a display screen for small characters involves going from a stem weight of one pixel to two pixels, the difference in weight between the two is 100%. In print, bold might typically be only 20 or 30 percent heavier than its equivalent regular or Roman face. Generally, this xe2x80x9cone pixel, two pixelxe2x80x9d problem has been treated as an inherent characteristic of display devices which must simply be accepted.
Prior work in the field of displaying characters has focused, in part, on the development of anti-aliasing technologies designed to improve the display of characters on CRT displays. A commonly used anti-aliasing technique involves using shades of gray for pixels which include edges of the character. In effect, this smudges shapes, reducing spatial frequency of the edges but better approximating the intended character shapes. While known anti-aliasing techniques can significantly improve the quality of characters displayed on a CRT display device, many of these techniques are ineffective when applied to LCD display devices which differ considerably from CRT displays in terms of pixel sub-component arrangement.
A further issue is that such anti-aliasing techniques work reasonably well on large sizes of text, but at the sizes most suited for human readingxe2x80x94typically between 8 and 13 pointsxe2x80x94they have the effect of blurring the text. For the human reader, the effect is similar to trying to read with someone else""s eyeglass prescription. The reader""s eye tries continually to focus on an image which is inherently unfocusable. This leads quickly to eyestrain and fatigue, making conventional anti-aliasing unsatisfactory in cases for sustained reading.
While anti-aliasing techniques have helped the aliasing problem associated with displaying relatively low resolution representations of text, at least on CRT displays, the problem of pixel size and the inability to accurately display character stem widths have, prior to the present invention, been considered a fixed characteristic of display devices which must be tolerated.
In view of the above, it is apparent that there is a need for new and improved methods and apparatus for displaying text on flat panel display devices. It is desirable that at least some of the new methods be suitable for use with existing display device and computers. It is also desirable that at least some methods and apparatus be directed to improving the quality of displayed text on new computers using, e.g., new display devices and/or new methods of displaying text.
While the display of text, which is a special case of graphics, is of major concern in many computer applications, there is also a need for improved methods and apparatus for displaying other graphics, geometric shapes, e.g., circles, squares, etc., and captured images such as photographs, accurately and clearly.
The present invention is directed to methods and apparatus for displaying images utilizing multiple distinct portions of an output device, e.g., an LCD display, to represent a single pixel of an image.
The inventors of the present application recognize the well-known principle that humans are much more sensitive to edges of luminance, where light intensity changes, than to edges of chrominance, where color intensity changes. This is why it is very difficult to read red text on a green background, for example. They also recognize the well-known principle that humans are not equally sensitive to the colors of red, green and blue. In fact, of 100 percent luminous intensity in a fully white pixel the red pixel sub-component contributes approximately 30% to the overall perceived luminance, green 60% and blue 10%.
Various features of the present invention are directed to utilizing the individual pixel sub-components of a display as independent luminous intensity sources thereby increasing the effective resolution of a display by as much as a factor of 3 in the dimension perpendicular to the direction of the RGB striping. This allows for a significant improvement in visible resolution.
While the methods of the present invention may result in some degradation in chrominance quality as compared to known display techniques, as discussed above humans are more sensitive to edges of luminance than of chrominance. Accordingly, the present invention can provide significant improvements in the quality of images, compared to known rendering techniques, even when taking into consideration the negative impact the techniques of the present invention may have on color quality.
As discussed above, known monitors tend to use vertical striping. Because character stems occur in the vertical direction the ability to accurately control the thickness of vertical lines when rendering horizontally flowing text tends to be more important than the ability to control the thickness of horizontal lines. With this in mind, it was concluded that, at least for text applications, it is often more desirable to have a monitor""s maximum resolution in the horizontal, as opposed to vertical direction. Accordingly, various display devices implemented in accordance with the present invention utilize vertical, as opposed to horizontal, RGB striping. This provides such monitors, when used in accordance with the present invention, greater resolution in the horizontal direction than in the vertical direction. The present invention can however be applied similarly to monitors with horizontal RGB striping resulting in improved resolution in the vertical direction as compared to conventional image rendering techniques.
In addition to new display devices which are suitable for use when treating pixel sub-components as independent luminous intensity sources, the present invention is directed to new and improved text, graphics and image rendering techniques which facilitate pixel sub-component use in accordance with the present invention.
The display of images, including text, involves several steps including, e.g., image scaling, hinting and scan conversion.
An image scaling technique of the present invention involves scaling geometric representations of text, in the dimension perpendicular to the direction of RGB striping, at a rate that is greater than the rate of scaling in the direction of RGB striping. Such a non-uniform scaling technique allows subsequent processing operations to take full advantage of the effective increase in resolution obtained by treating pixel sub-components as individual luminous intensity sources. Scaling in the direction perpendicular to the striping may also be made a function of one or more weighting factors used in a subsequent scan conversion operation. Accordingly scaling in the direction perpendicular to the striping may be many times, e.g., 10 times, the scaling performed in the direction of the striping.
In addition to new scaling methods, the present invention is directed to new methods of performing hinting operations. These methods take into consideration pixel sub-component boundaries within an image, in addition to pixel boundaries considered in known hinting operations. Some hinting operations performed for use with display devices with vertical striping involve, as a step, aligning characters along pixel sub-component boundaries so that the character stem borders on, or even within, a red, blue or green pixel sub-component, as opposed to always between blue and red pixel sub-components as occurs at the whole pixel edge.
Other hinting operations may be performed for use with display devices with horizontal striping. Such hinting operations involve as a step, aligning character bases along pixel sub-component boundaries so that the character bases border on or within red or blue pixel sub-components, as opposed to a whole pixel edge.
In accordance with the present invention, as part of a hinting operation the width of vertical and/or horizontal lines within an image may be adjusted as a function of pixel sub-component boundaries. This allows for the hinting processes to perform finer adjustments when distorting an image""s shape than in known systems where hinting is performed as a function of the location of whole pixel boundaries (edges) as opposed to pixel sub-component boundaries.
Scan conversion normally follows hinting. Scan conversion is the process by which geometric representations of images are converted into bitmaps. Scan conversion operations of the present invention involve mapping different portions of an image into different pixel sub-components. This differs significantly from known scan conversion techniques where the same portion of an image is used to determine the luminous intensity values to be used with each of the three pixel sub-components which represent a pixel.
As a result of treating RGB pixel sub-components as independent luminous intensity sources, color fringing effects may be encountered. One feature of the present invention is directed to processing bitmapped images to detect undesirable color fringing effects. Another feature of the invention is directed to performing color processing operations on bitmapped images to lessen or compensate for undesirable color fringing effects.
Other features of the present invention are directed to gray scale displays, and displays capable of displaying gray scale images, e.g., text, and color images.
The present invention includes a gray scale monitor with rectangular display elements that provide a resolution in a first dimension, e.g., the horizontal dimension, which is a multiple of the resolution in a second dimension perpendicular to the first dimension, e.g., the vertical direction. In one exemplary embodiment, a gray scale display is provided which has 3 times the resolution in the horizontal dimension than it has in the vertical dimension.
Displays, e.g., LCD displays, which are capable of operating as both gray scale and color display devices are also contemplated. In one such display, the color filter used to implement a color portion of the display is omitted from a separate gray scale portion of the display. Text, e.g., captions, are displayed using the gray scale portion of the display while color images, e.g., graphics, are displayed using the color portion of the display.
In another display of the present invention, a color filter with filter cells that can be switched between a color and a clear mode of operation are employed. When images, e.g., text, are to be displayed as gray scale images, the filter cells, corresponding to the portion of the display to be used to display the gray scale images, are switched to the clear mode of operation. In such an embodiment, the remaining portion or portions of the display may be used to display color images.
Numerous additional features, embodiments, and advantages of the methods and apparatus of the present invention are set forth in the detailed description which follows.